Explicit Neural Word Representation

Recent advances in word embedding provide significant benefit to various information processing tasks. Yet these dense representations and their estimation of word-to-word relatedness remain difficult to interpret and hard to analyze. As an alternative, explicit word representations propose vectors whose dimensions are easily interpretable, and recent methods show competitive performance to the dense vectors. We introduce a neural-based explicit representation, rooted in the conceptual ideas of the word2vec Skip-Gram model. The method provides interpretable explicit vectors while keeping the effectiveness of the Skip-Gram model. The evaluation of various explicit representations on word association collections shows that the newly proposed method outperforms the state-of-the-art explicit representations when tasked with ranking highly similar terms. As a case study on the use of our explicit representation, we show the degree of the existence of gender bias in the English language (used in Wikipedia) in regards to various occupations. By measuring the bias towards explicit Female and Male factors, the study quantifies a general tendency of the majority of the occupations to male and a strong bias in a few specific occupations (e.g. nurse) to female. ACM Reference format: Navid Rekabsaz, Mihai Lupu, Allan Hanbury and Bhaskar Mitra. 2017. Explicit Neural Word Representation. In Proceedings of ACM International Conference on Information and Knowledge Management, Singapore, 2017